Data processing systems often comprise service frameworks in which program processes send service requests to services, and the services subsequently reply with responses. For example, a process may request certain data from a service, and the service may respond with the requested data. In some situations, program processes may make many service requests in parallel and wait for responses from each of multiple different services.
Various types of events may occur within a service framework that may lead to a degraded performance mode. A degraded performance mode may involve a situation in which a service is merely troubled, a situation in which the service has failed, or a situation in which some other problem exists. For example, a service may become overloaded (causing intermittent or slow responses), may become completely inoperative, may become unreachable due to network problems, and so on. In the event of a failure mode, the process may slow to a standstill as it continues to wait for the response from the service. Typically, it may not be known whether the service has failed or whether it is merely slow in returning a response. At some point, a decision needs to be made as to how long the process should wait for the response from the service.
Accordingly, a problem that has been encountered in the context of service frameworks is to provide a mechanism for program processes to decide how long to wait for a response from a service in situations where the service may have entered a failure mode or other degraded performance mode. Also, oftentimes, services enter a failure mode as a result of becoming overloaded. Accordingly, another problem that has been encountered is to provide a mechanism that avoids services becoming overloaded. It will be appreciated that while the teachings herein describe certain features and advantages that may be achieved, the teachings herein may be used to implement systems and methods that do not necessarily have any of these features or advantages, but rather have other features and advantages.